Adding Llama Index Embedding Instrumentation on the basis of LLMInvocation #83
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add embedding event handlers (_handle_embedding_start, _handle_embedding_end) - Extract model name, input texts, and dimension count from embedding events - Create vendor_detection.py module with VendorRule-based provider detection - Support 13+ embedding providers (OpenAI, Azure, AWS, Google, Cohere, etc.) - Add test_embedding_instrumentation.py with single and batch embedding tests - Update README with embedding documentation and provider list - Tested successfully with OpenAI embeddings API
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Embedding Instrumentation
Tracks embedding generation events with input texts and output dimensions
Vendor detection system supporting 13+ providers:
OpenAI, Azure OpenAI, AWS Bedrock, Google (Vertex AI, Gemini)
Cohere, Anthropic, Hugging Face, Ollama
Voyage AI, Jina, Mistral, and more
Rule-based provider identification from class names
Batch and single embedding operation support
Infrastructure
LlamaindexInstrumentor class with automatic callback handler registration
LlamaindexCallbackHandler implementing LlamaIndex event handling
vendor_detection.py module with extensible VendorRule system
Integration with opentelemetry-util-genai for span and metrics emission
Comprehensive test coverage (338 lines of tests)
Testing
✅ LLM instrumentation: 187 lines of tests
✅ Embedding instrumentation: 151 lines of tests
✅ Tested with OpenAI embeddings API
✅ All linting and formatting checks pass